CLAIMS 



What is claimed is: 



1 1 . A method of modeling a complex process having a plurality of input variables, a 

2 portion of which have unknown behavior that can be described by a function comprising 

3 at least one unknown parameter and producing an output that is a predictor of outcome of 

4 the process, the method comprising the steps of: 

5 providing a non-linear regression model of the process comprising: 

6 a plurality of first connection weights that relate the plurality of input 

7 variables to a plurality of process metrics; and 

8 a function and a plurality of second connection weights that relate input 

9 variables in the portion to the plurality of process metrics, wherein each of the plurality of 

10 second connection weights correspond to an unknown parameter associated with an input 

1 1 variable in the portion; and 

12 using the model to predict an outcome of the process. 

1 2. The method of claim 1 , wherein the model has at least a first hidden layer and a 

2 last hidden layer, the first hidden layer having a plurality of nodes each corresponding to 

3 input variables in the portion, each node in the first hidden layer relating to an input 

4 variable with the function and a second connection weight, the second connection weight 

5 corresponding to the at least one unknown parameter. 

1 3. The method of claim 2, wherein the last hidden layer is connected to nodes in the 

2 first hidden layer and nodes associated with input variables that are not in the portion. 

1 4. The method of claim 3, wherein the function comprises two unknown parameters 

2 and can be represented by a first function with a first unknown parameter and a second 

3 function with a second unknown parameter, the method further comprising: 

4 providing a non-linear regression model of the process comprising: 

5 a first hidden layer, a second hidden layer, and a last hidden layer, the 

6 second hidden layer having a plurality of nodes each corresponding to one of the plurality 

7 of nodes in the first hidden layer, 

8 a first function and a plurality of second connection weights that relate 

9 input variables in the portion to nodes in the first hidden layer, wherein each of the 

10 plurality of second connection weights correspond to a first unknown parameter 

1 1 associated with an input variable in the portion; 
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1 2 a second function and a plurality of third connection weights that relate 

1 3 nodes in the first hidden layer to nodes in the second hidden layer, wherein each of the 

14 plurality of third connection weights correspond to a second unknown parameter 

1 5 associated with an input variable in the portion; and 

16 a plurality of first connection weights that relate the plurality of input 

1 7 variables not in the portion and nodes in the second hidden layer to a plurality of process 

1 8 metrics. 

1 5. The method of claim 1 , wherein the function is non-linear with respect to the 

2 input variable. 

1 6. The method of claim 5, wherein the input variable represents a time elapsed since 

2 an event associated with the complex process. 

1 7. The method of claim 1 , wherein the input variables in the portion of the plurality 

2 of input variables are maintenance variables of a complex manufacturing process and the 

3 other input variables are manipulable variables. 

1 8. The method of claim 1 , wherein the function is an activation function of the form 

3 where Xj is the synaptic weight associated with an input and the input yj is an input 

4 variable in the portion. 

1 9. The method of claim 8, wherein the input yj represents a time elapsed since a 

2 maintenance event. 

1 10. The method of claim 1 , wherein the input variable comprises a discrete value. 

1 11. A method of building a non-linear regression model of a complex process having 

2 a plurality of input variables, a portion of which have unknown behavior that can be 

3 described by a function comprising at least one unknown parameter and producing an 

4 output that is a predictor of outcome of the complex process, the method comprising the 

5 steps of: 

6 (a) identifying the function; 

7 (b) providing a model comprising a plurality of connection weights that relate 

8 the plurality of input variables to a plurality of process metrics; 
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9 (c) determining an error signal for the model; 

1 0 (d) adjusting the one or more unknown parameters of the function and the 

1 1 plurality of connection weights in a single process based on the error signal; and 

1 2 (e) repeating steps (c) and (d) until a convergence criterion is satisfied. 

1 12. The method of claim 1 1 wherein: 

2 a portion of the input variables are input variables for a first hidden layer of the 

3 non-linear regression model, the first hidden layer having a plurality of nodes each 

4 associated with one of the input variables of the portion and having a single synaptic 

5 weight; 

6 the identified function relates to an input variable from the portion; 

7 the error signal is determined for an output layer of the non-linear regression 

8 model; and 

9 the error signal is used to determine a gradient for a plurality of outputs of the first 
10 hidden layer. 

1 13. The method of claim 1 1 , wherein the function is non-linear with respect to the 

2 input variable. 

1 1 4. The method of claim 13, wherein the input variable represents a time elapsed 

2 since an event associated with the complex process. 

1 15. The method of claim 1 1 , wherein the input variable in the portion of the plurality 

2 of input variables are maintenance variables of a complex manufacturing process. 

1 16. The method of claim 1 1 , wherein the function is an activation function of the 

2 form 

3 expiry j) 

4 where Aj is the synaptic weight associated with an input y j9 and the input yj is an input 

5 variable of the portion of the plurality input variables. 

1 17. The method of claim 1 6, wherein the adjustment is of the form 

2 AXj = -TiyjSj 

3 where r\ is a learning rate parameter, Sj is the gradient of an output of a node ; of the first 

4 hidden layer with the input y b A/l y is the adjustment for synaptic weight A associated with 
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the input y Jy and the input yj is an input variable of the portion of the plurality input 
variables. 



1 18. An article of manufacture comprising a computer-readable medium having 

2 computer-readable instructions for 

3 determining an error signal for an output layer of a non-linear regression model of 

4 a complex process, the model having a plurality of input variables of which a portion are 

5 input variables for a first hidden layer of the model having a plurality of nodes, each node 

6 associated with one of the input variables of the portion and having a single synaptic 

7 weight; 

8 using the error signal to determine a gradient for a plurality of outputs of the first 

9 hidden layer; 

10 determining an adjustment to one or more of the synaptic weights corresponding 

11 to one or more unknown parameters of a function; and 

1 2 evaluating a convergence criterion and repeating foregoing steps if the 

1 3 convergence criterion is not satisfied, 

14 wherein the computer-readable medium is in signal communication with a 

1 5 memory device for storing the function and the one or more synaptic weights. 

1 19. An article of manufacture for building a non-linear regression model of a complex 

2 process having a plurality of input variables, a portion of which have unknown behavior 

3 that can be described by a function comprising at least one unknown parameter and 

4 producing an output that is a predictor of outcome of the complex process, the article of 

5 manufacture comprising: 

6 a process monitor for providing training data representing a plurality of input 

7 variables and a plurality of corresponding process metrics; 

8 a memory device for providing the function and a plurality of first weights 

9 corresponding to the at least one unknown parameter associated with each of the plurality 

1 0 of input variables in the portion; and 

1 1 a data processing device in signal communication with the process monitor and 

12 the memory device, the data processing device receiving the training data, the function, 

13 and the plurality of first weights, determining an error signal for the non-linear regression 

14 model; and adjusting (i) the plurality of first weights and (ii) a plurality of second weights 
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1 5 that relate the plurality of input variables to the plurality of process metrics, in a single 

1 6 process based on the error signal. 

1 20. The article of manufacture of claim 19, wherein the function is non-linear with 

2 respect to the input variable. 

1 21. The article of manufacture of claim 1 9, wherein the function is an activation 

2 function of the form 

exp(-V,.) 

4 and wherein the adjustment is of the form 

5 A?ij = -r|yj5j 

6 where A 7 is the synaptic weight associated with an input y b the input yj is an input variable 

7 in the portion, r| is a learning rate parameter, § is the gradient of an output of a node; of 

8 the first hidden layer with the input y Jt and AAj is the adjustment for synaptic weight X } 

9 associated with the input yj. 

1 22. The article of manufacture of claim 1 9 wherein the data processing device further 

2 determines if a convergence criterion is satisfied. 

1 23 . The article of manufacture of claim 1 9 wherein the process monitor comprises a 

2 database. 

1 24. The article of manufacture of claim 19 wherein the process monitor comprises a 

2 memory device including a plurality of data files, each data file comprising a plurality of 

3 scalar numbers representing associated values for the plurality of input variables and the 

4 plurality of corresponding process metrics. 

1 25 . An article of manufacture for modeling a complex process having a plurality of 

2 input variables, a portion of which have unknown behavior that can be described by a 

3 function comprising at least one unknown parameter and producing an output that is a 

4 predictor of outcome of the complex process, the article of manufacture comprising: 

5 a process monitor for providing a plurality of input variables; 

6 a memory device for providing a plurality of first connection weights that relate 

7 the plurality of input variables to a plurality of process metrics, the function, and a 

8 plurality of second connection weights corresponding to the at least one unknown 
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9 parameter associated with each of the plurality of input variables in the portion; and 

1 0 a data processing device in signal communication with the process monitor and 

1 1 the memory device, the data processing device receiving the plurality of input variables, 

12 the plurality of first connection weights, the function, and the plurality of second 

1 3 connection weights; and predict an outcome of the process in a single process using the 

14 plurality of input variables, the plurality of first connection weights, the function, and the 

1 5 plurality of second connection weights. 
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